{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Orchestrating Jobs, Model Registration, and Continuous Deployment with Amazon SageMaker\n",
    "\n",
    "Amazon SageMaker offers Machine Learning application developers and Machine Learning operations engineers the ability to orchestrate SageMaker jobs and author reproducible Machine Learning pipelines, deploy custom-build models for inference in real-time with low latency or offline inferences with Batch Transform, and track lineage of artifacts. You can institute sound operational practices in deploying and monitoring production workflows, deployment of model artifacts, and track artifact lineage through a simple interface, adhering to safety and best-practice paradigmsfor Machine Learning application development.\n",
    "\n",
    "The SageMaker Pipelines service supports a SageMaker Machine Learning Pipeline Domain Specific Language (DSL), which is a declarative Json specification. This DSL defines a Directed Acyclic Graph (DAG) of pipeline parameters and SageMaker job steps. The SageMaker Python Software Developer Kit (SDK) streamlines the generation of the pipeline DSL using constructs that are already familiar to engineers and scientists alike.\n",
    "\n",
    "The SageMaker Model Registry is where trained models are stored, versioned, and managed. Data Scientists and Machine Learning Engineers can compare model versions, approve models for deployment, and deploy models from different AWS accounts, all from a single Model Registry. SageMaker enables customers to follow the best practices with ML Ops and getting started right. Customers are able to standup a full ML Ops end-to-end system with a single API call."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## SageMaker Pipelines\n",
    "\n",
    "Amazon SageMaker Pipelines support the following activites:\n",
    "\n",
    "* Pipelines - A Directed Acyclic Graph of steps and conditions to orchestrate SageMaker jobs and resource creation.\n",
    "* Processing Job steps - A simplified, managed experience on SageMaker to run data processing workloads, such as feature engineering, data validation, model evaluation, and model interpretation.\n",
    "* Training Job steps - An iterative process that teaches a model to make predictions by presenting examples from a training dataset.\n",
    "* Conditional step execution - Provides conditional execution of branches in a pipeline.\n",
    "* Registering Models - Creates a model package resource in the Model Registry that can be used to create deployable models in Amazon SageMaker.\n",
    "* Creating Model steps - Create a model for use in transform steps or later publication as an endpoint.\n",
    "* Parametrized Pipeline executions - Allows pipeline executions to vary by supplied parameters.\n",
    "* Transform Job steps - A batch transform to preprocess datasets to remove noise or bias that interferes with training or inference from your dataset, get inferences from large datasets, and run inference when you don't need a persistent endpoint."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Layout of the SageMaker ModelBuild Project Template\n",
    "\n",
    "The template provides a starting point for bringing your SageMaker Pipeline development to production.\n",
    "\n",
    "```\n",
    "sagemaker-project-modelbuild/\n",
    "|-- codebuild-buildspec.yml\n",
    "|-- CONTRIBUTING.md\n",
    "|-- pipelines\n",
    "|   |-- dsoaws\n",
    "|   |   |-- evaluate.py\n",
    "|   |   |-- __init__.py\n",
    "|   |   |-- pipeline.py\n",
    "|   |   `-- preprocess.py\n",
    "|   |-- get_pipeline_definition.py\n",
    "|   |-- __init__.py\n",
    "|   |-- run_pipeline.py\n",
    "|   |-- _utils.py\n",
    "|   `-- __version__.py\n",
    "|-- README.md\n",
    "|-- setup.cfg\n",
    "|-- setup.py\n",
    "|-- tests\n",
    "|   `-- test_pipelines.py\n",
    "`-- tox.ini\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A description of some of the artifacts is provided below:\n",
    "<br/><br/>\n",
    "Your codebuild execution instructions:\n",
    "```\n",
    "sagemaker-project-modelbuild/\n",
    "|-- codebuild-buildspec.yml\n",
    "```\n",
    "<br/><br/>\n",
    "Your pipeline artifacts, which includes a pipeline module defining the required `get_pipeline` method that returns an instance of a SageMaker pipeline, a preprocessing script that is used in feature engineering, and a model evaluation script to measure the accuracy of the trained model:\n",
    "\n",
    "```\n",
    "|-- pipelines\n",
    "|   |-- dsoaws\n",
    "|   |   |-- evaluate_metrics.py\n",
    "|   |   |-- __init__.py\n",
    "|   |   |-- pipeline.py\n",
    "|   |   `-- preprocess-scikit-text-to-bert-feature-store.py\n",
    "```\n",
    "<br/><br/>\n",
    "Utility modules for getting pipeline definition jsons and running pipelines:\n",
    "\n",
    "```\n",
    "|-- pipelines\n",
    "|   |-- get_pipeline_definition.py\n",
    "|   |-- __init__.py\n",
    "|   |-- run_pipeline.py\n",
    "|   |-- _utils.py\n",
    "|   `-- __version__.py\n",
    "```\n",
    "<br/><br/>\n",
    "Python package artifacts:\n",
    "\n",
    "```\n",
    "|-- setup.cfg\n",
    "|-- setup.py\n",
    "```\n",
    "<br/><br/>\n",
    "A stubbed testing module for testing your pipeline as you develop:\n",
    "\n",
    "```\n",
    "|-- tests\n",
    "|   `-- test_pipelines.py\n",
    "```\n",
    "<br/><br/>\n",
    "The `tox` testing framework configuration:\n",
    "\n",
    "```\n",
    "`-- tox.ini\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### A SageMaker Pipeline\n",
    "\n",
    "The pipeline that we create follows a typical Machine Learning Application pattern of pre-processing, training, evaluation, and conditional model registration and publication, if the quality of the model is sufficient.\n",
    "\n",
    "![A typical ML Application pipeline](../img/pipeline-full.png)\n",
    "\n",
    "### Getting some constants\n",
    "\n",
    "We get some constants from the local execution environment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import boto3\n",
    "import sagemaker\n",
    "import pandas as pd\n",
    "\n",
    "sess   = sagemaker.Session()\n",
    "default_bucket = sess.default_bucket()\n",
    "role = sagemaker.get_execution_role()\n",
    "region = boto3.Session().region_name\n",
    "\n",
    "sm = boto3.Session().client(service_name='sagemaker', region_name=region)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(role)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%store -r sagemaker_project_name\n",
    "%store -r sagemaker_project_name_and_id\n",
    "%store -r sagemaker_mlops_build_code"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print('Project name: {}'.format(sagemaker_project_name))\n",
    "print('Project name and id: {}'.format(sagemaker_project_name_and_id))\n",
    "print('Source code for our mlops build pipeline: {}'.format(sagemaker_mlops_build_code))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Get a Pipeline using our Cloned Repository\n",
    "\n",
    "Here we get the pipeline instance from your pipeline module so that we can work with it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sys\n",
    "\n",
    "sys.path.insert(0, sagemaker_mlops_build_code)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "########\n",
    "## NOTE:  IF YOU MAKE CHANGES TO get_pipeline, \n",
    "#         YOU NEED TO MAKE THE CHANGE TO THE CLONED REPO\n",
    "#         AT `sagemaker_mlops_build_code`\n",
    "########\n",
    "\n",
    "from pipelines.dsoaws.pipeline import get_pipeline\n",
    "\n",
    "pipeline = get_pipeline(\n",
    "    pipeline_name=sagemaker_project_name_and_id,    \n",
    "    region=region,\n",
    "    role=role,\n",
    "    default_bucket=default_bucket,\n",
    "    model_package_group_name=sagemaker_project_name_and_id,\n",
    "    base_job_prefix=sagemaker_project_name_and_id\n",
    ")\n",
    "\n",
    "print(pipeline)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pipeline.definition()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# AT THIS POINT, YOUR PIPELINE IS ALREADY EXECUTING\n",
    "If you run the following code, you will start ANOTHER pipeline execution."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Start Pipeline Execution\n",
    "\n",
    "We can run additional executions of the pipeline specifying different pipeline parameters. The parameters argument is a dictionary whose names are the parameter names, and whose values are the primitive values to use as overrides of the defaults.\n",
    "\n",
    "Of particular note, based on the performance of the model, we may want to kick off another pipeline execution, but this time on a compute-optimized instance type and set the model approval status automatically be \"Approved\". This means that the model package version generated by the `RegisterModel` step will automatically be ready for deployment through CI/CD pipelines, such as with SageMaker Projects."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "raw_input_data_s3_uri = 's3://{}/amazon-reviews-pds/tsv/'.format(default_bucket)\n",
    "print(raw_input_data_s3_uri)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# IGNORE THE FOLLOWING ERROR:\n",
    "```\n",
    "No finished training job found associated with this estimator. Please make sure this estimator is only used for building workflow config\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(pipeline)\n",
    "\n",
    "execution = pipeline.start(\n",
    "    parameters=dict(\n",
    "        InputDataUrl=raw_input_data_s3_uri,\n",
    "        ProcessingInstanceCount=1,\n",
    "        ProcessingInstanceType='ml.c5.2xlarge',\n",
    "        MaxSeqLength=64,\n",
    "        BalanceDataset='True',\n",
    "        TrainSplitPercentage=0.9,\n",
    "        ValidationSplitPercentage=0.05,\n",
    "        TestSplitPercentage=0.05,\n",
    "#        FeatureStoreOfflinePrefix='', # TODO:  pass this in or leave empty for default?\n",
    "#        FeatureGroupName='', # TODO:  pass this in or leave empty for default?\n",
    "        TrainingInstanceType='ml.c5.9xlarge',\n",
    "        TrainingInstanceCount=1,\n",
    "        Epochs=1,\n",
    "        LearningRate=0.00001,\n",
    "        Epsilon=0.00000001,\n",
    "        TrainBatchSize=128,\n",
    "        ValidationBatchSize=128,\n",
    "        TestBatchSize=128,\n",
    "        TrainStepsPerEpoch=50,\n",
    "        ValidationSteps=50,\n",
    "        TestSteps=50,\n",
    "        TrainVolumeSize=1024,\n",
    "        UseXLA='True',\n",
    "        UseAMP='True',\n",
    "        FreezeBERTLayer='False',\n",
    "        EnableSageMakerDebugger='False',\n",
    "        EnableCheckpointing='False',\n",
    "        EnableTensorboard='False',\n",
    "        InputMode='File',\n",
    "        RunValidation='True',\n",
    "        RunTest='Fasle',\n",
    "        RunSamplePredictions='False', \n",
    "        MinAccuracyValue=0.01,\n",
    "        ModelApprovalStatus='Approved', # TODO:  Figure out why this is not working\n",
    "        DeployInstanceType='ml.m5.4xlarge',\n",
    "        DeployInstanceCount=1\n",
    "    )\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "execution_run = execution.describe()\n",
    "print(execution_run)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "execution_run_name = execution_run['PipelineExecutionDisplayName']\n",
    "print(execution_run_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# List Execution Steps"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "execution.list_steps()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "execution.wait()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "execution.list_steps()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# List All Artifacts Generated By The Pipeline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "processing_job_name=None\n",
    "training_job_name=None"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import time\n",
    "from sagemaker.lineage.visualizer import LineageTableVisualizer\n",
    "\n",
    "viz = LineageTableVisualizer(sagemaker.session.Session())\n",
    "for execution_step in reversed(execution.list_steps()):\n",
    "    print(execution_step)\n",
    "    # We are doing this because there appears to be a bug of this LineageTableVisualizer handling the Processing Step\n",
    "    if execution_step['StepName'] == 'Processing':\n",
    "        processing_job_name=execution_step['Metadata']['ProcessingJob']['Arn'].split('/')[-1]\n",
    "        print(processing_job_name)\n",
    "        display(viz.show(processing_job_name=processing_job_name))\n",
    "    elif execution_step['StepName'] == 'Train':\n",
    "        training_job_name=execution_step['Metadata']['TrainingJob']['Arn'].split('/')[-1]\n",
    "        print(training_job_name)\n",
    "        display(viz.show(training_job_name=training_job_name))\n",
    "    else:\n",
    "        display(viz.show(pipeline_execution_step=execution_step))\n",
    "        time.sleep(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Create the Experiment \n",
    "\n",
    "We already create the Experiment in run_pipeline.py."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# import time\n",
    "# from smexperiments.experiment import Experiment\n",
    "\n",
    "# timestamp = int(time.time())\n",
    "\n",
    "# experiment = Experiment.create(\n",
    "#                 experiment_name=pipeline_name,\n",
    "#                 description='Amazon Customer Reviews BERT Pipeline Experiment', \n",
    "#                 sagemaker_boto_client=sm)\n",
    "\n",
    "# experiment_name = experiment.experiment_name\n",
    "# print('Experiment name: {}'.format(experiment_name))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Associate the Pipeline Execution with our Experiment"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "import time\n",
    "from smexperiments.trial import Trial\n",
    "\n",
    "trial = Trial.create(trial_name=execution_run_name,\n",
    "                     experiment_name=pipeline_name,\n",
    "                     sagemaker_boto_client=sm)\n",
    "\n",
    "trial_name = trial.trial_name\n",
    "print('Trial name: {}'.format(trial_name))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Add Processing Job and Training Job to our Experiment"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# -aws-processing-job is the default name assigned by ProcessingJob\n",
    "processing_job_tc = '{}-aws-processing-job'.format(processing_job_name)\n",
    "\n",
    "print(processing_job_tc)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "response = sm.associate_trial_component(\n",
    "    TrialComponentName=processing_job_tc,\n",
    "    TrialName=trial_name\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# -aws-training-job is the default name assigned by TrainingJob\n",
    "training_job_tc = '{}-aws-training-job'.format(training_job_name)\n",
    "\n",
    "print(training_job_tc)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "response = sm.associate_trial_component(\n",
    "    TrialComponentName=training_job_tc,\n",
    "    TrialName=trial_name\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Log Additional Parameters within Trial"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from smexperiments import tracker\n",
    "\n",
    "processing_job_tracker = tracker.Tracker.load(trial_component_name=processing_job_tc)\n",
    "print(processing_job_tracker)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "processing_job_tracker.log_parameters({\n",
    "    \"balance_dataset\": str(balance_dataset), \n",
    "})\n",
    "\n",
    "# must save after logging\n",
    "processing_job_tracker.trial_component.save()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Analyze Experiment"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sagemaker.analytics import ExperimentAnalytics\n",
    "\n",
    "import pandas as pd\n",
    "pd.set_option(\"max_colwidth\", 500)\n",
    "#pd.set_option(\"max_rows\", 100)\n",
    "\n",
    "experiment_analytics = ExperimentAnalytics(\n",
    "    experiment_name=pipeline_name,\n",
    ")\n",
    "\n",
    "experiment_analytics_df = experiment_analytics.dataframe()\n",
    "experiment_analytics_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Download the Trained Model\n",
    "Fix these parameters using the sagemaker_project_name_and_id"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "training_job_arn=None\n",
    "\n",
    "for execution_step in execution.list_steps():\n",
    "    if execution_step[\"StepName\"] == \"TrainModel\":\n",
    "        training_job_arn = execution_step[\"Metadata\"][\"TrainingJob\"][\"Arn\"]\n",
    "        break\n",
    "\n",
    "print(training_job_arn)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "training_job_name = training_job_arn.split('/')[-1]\n",
    "print(training_job_name)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!aws s3 cp s3://$default_bucket/$sagemaker_project_name_and_id/output/model/$training_job_name/output/model.tar.gz ./\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!tar -C ./downloaded_model -zxvf model.tar.gz"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!ls -al ./downloaded_model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Approve the Registered Model for Staging\n",
    "The pipeline that was executed created a Model Package version within the specified Model Package Group. Of particular note, the registration of the model/creation of the Model Package was done so with approval status as `PendingManualApproval`.\n",
    "\n",
    "Notes:  \n",
    "* You can do this within SageMaker Studio, as well.  However, we are approving programmatically here in this example.\n",
    "* This approval is only for Staging.  For Production, you must go through the CodePipeline (deep link is below.)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_package_arn=None\n",
    "\n",
    "for execution_step in execution.list_steps():\n",
    "    if execution_step[\"StepName\"] == \"RegisterModel\":\n",
    "        model_package_arn = execution_step[\"Metadata\"][\"RegisterModel\"][\"Arn\"]\n",
    "        break\n",
    "\n",
    "print(model_package_arn)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "model_package_update_response = sm.update_model_package(\n",
    "    ModelPackageArn=model_package_arn,\n",
    "    ModelApprovalStatus=\"Approved\",\n",
    ")\n",
    "\n",
    "print(model_package_update_response)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Check CodePipeline Deployment"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from IPython.core.display import display, HTML\n",
    "\n",
    "display(HTML('<b>Check Deployment in <a target=\"blank\" href=\"https://console.aws.amazon.com/codesuite/codepipeline/pipelines/sagemaker-{}-modeldeploy/view?region={}\">CodePipeline</a></b>'.format(sagemaker_project_name_and_id, region)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Check the Staging Endpoint"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "staging_endpoint_name='{}-staging'.format(sagemaker_project_name)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from IPython.core.display import display, HTML\n",
    "\n",
    "display(HTML('<b>Review <a target=\"blank\" href=\"https://console.aws.amazon.com/sagemaker/home?region={}#/endpoints/{}\">SageMaker Staging REST Endpoint</a></b>'.format(region, staging_endpoint_name)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Run Predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import boto3\n",
    "import sagemaker\n",
    "import pandas as pd\n",
    "\n",
    "sess   = sagemaker.Session()\n",
    "default_bucket = sess.default_bucket()\n",
    "role = sagemaker.get_execution_role()\n",
    "region = boto3.Session().region_name\n",
    "\n",
    "sm = boto3.Session().client(service_name='sagemaker', region_name=region)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# _Wait Until the Staging Endpoint is Deployed_\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "\n",
    "waiter = sm.get_waiter('endpoint_in_service')\n",
    "waiter.wait(EndpointName=staging_endpoint_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# _Wait Until the ^^ Staging Endpoint ^^ is Deployed_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Run A Sample Prediction in Staging"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sagemaker.tensorflow.model import TensorFlowPredictor\n",
    "\n",
    "predictor = TensorFlowPredictor(endpoint_name=staging_endpoint_name,\n",
    "                                sagemaker_session=sess,\n",
    "                                model_name='saved_model',\n",
    "                                model_version=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "reviews = [\"This is not great!\"]\n",
    "\n",
    "predicted_classes = predictor.predict(reviews)\n",
    "\n",
    "for predicted_class, review in zip(predicted_classes, reviews):\n",
    "    print('[Predicted Star Rating: {}]'.format(predicted_class), review)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Approve Staging Model for Production\n",
    "![](../img/approve-model-staging-to-production.png)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "from IPython.core.display import display, HTML\n",
    "\n",
    "display(HTML('<b>Approve Model for Production in <a target=\"blank\" href=\"https://console.aws.amazon.com/codesuite/codepipeline/pipelines/sagemaker-{}-modeldeploy/view?region={}\">CodePipeline</a></b>'.format(sagemaker_project_name_and_id, region)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Get the Production Endpoint Name"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "production_endpoint_name='{}-prod'.format(sagemaker_project_name)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from IPython.core.display import display, HTML\n",
    "\n",
    "display(HTML('<b>Review <a target=\"blank\" href=\"https://console.aws.amazon.com/sagemaker/home?region={}#/endpoints/{}\">SageMaker Production REST Endpoint</a></b>'.format(region, production_endpoint_name)))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# _Wait Until the Production Endpoint is Deployed_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "\n",
    "waiter = sm.get_waiter('endpoint_in_service')\n",
    "waiter.wait(EndpointName=production_endpoint_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# _Wait Until the ^^ Production Endpoint ^^ is Deployed_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sagemaker.tensorflow.model import TensorFlowPredictor\n",
    "\n",
    "predictor = TensorFlowPredictor(endpoint_name=production_endpoint_name,\n",
    "                                sagemaker_session=sess,\n",
    "                                model_name='saved_model',\n",
    "                                model_version=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "reviews = [\"This is great!\", \n",
    "           \"This is not good.\"]\n",
    "\n",
    "predicted_classes = predictor.predict(reviews)\n",
    "\n",
    "for predicted_class, review in zip(predicted_classes, reviews):\n",
    "    print('[Predicted Star Rating: {}]'.format(predicted_class), review)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Release Resources"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%html\n",
    "\n",
    "<p><b>Shutting down your kernel for this notebook to release resources.</b></p>\n",
    "<button class=\"sm-command-button\" data-commandlinker-command=\"kernelmenu:shutdown\" style=\"display:none;\">Shutdown Kernel</button>\n",
    "        \n",
    "<script>\n",
    "try {\n",
    "    els = document.getElementsByClassName(\"sm-command-button\");\n",
    "    els[0].click();\n",
    "}\n",
    "catch(err) {\n",
    "    // NoOp\n",
    "}    \n",
    "</script>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%javascript\n",
    "\n",
    "try {\n",
    "    Jupyter.notebook.save_checkpoint();\n",
    "    Jupyter.notebook.session.delete();\n",
    "}\n",
    "catch(err) {\n",
    "    // NoOp\n",
    "}"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (Data Science)",
   "language": "python",
   "name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:us-east-1:081325390199:image/datascience-1.0"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
